Variable interaction in multi-objective optimization problems

14th International Conference on Parallel Problem Solving from Nature – PPSN XIV, 2016-09-17, 2016-09-21, Edinburgh, UK, pp. 399 - 409This is the author accepted manuscript. The final version is available from the publisher via the DOI in this record.The final publication is available at link.springer.comVariable interaction is an important aspect of a problem, which reflects its structure, and has implications on the design of efficient optimization algorithms. Although variable interaction has been widely studied in the global optimization community, it has rarely been explored in the multi-objective optimization literature. In this paper, we empirically and analytically study the variable interaction structures of some popular multi-objective benchmark problems. Our study uncovers nontrivial variable interaction structures for the ZDT and DTLZ benchmark problems which were thought to be either separable or non-separable

Contribution based multi-island competitive cooperative coevolution

Competition in cooperative coevolution (CC) has demonstrated success in solving global optimization problems. In a recent study, a multi-island competitive cooperative coevolution (MIC3) algorithm was introduced that featured competition and collaboration of several different problem decomposition strategies implemented as independent islands. It was shown that MIC3converges to high quality solutions without the need to find an optimal decomposition. MIC3splits the computational budget in terms of the number of function evaluations, equally amongst all the islands and evolves them in a round-robin fashion. This overlooks the difference in contributions of different islands towards improving the overall objective function value. Therefore, a considerable amount of function evaluations is wasted on the low-contributing islands as their problem decomposition strategies may not appeal to the problem at the given stage of the evolutionary process. This paper proposes contribution-based MIC3 algorithms (MIC4) that quantifies the contributions of each island and allocates the computational budget accordingly. The experimental analysis reveals that the proposed method outperforms its counterpart

CBCC3 — A contribution-based cooperative co-evolutionary algorithm with improved exploration/exploitation balance

Cooperative Co-evolution (CC) is a promising framework for solving large-scale optimization problems. However, the round-robin strategy of CC is not an efficient way of allocating the available computational resources to components of imbalanced functions. The imbalance problem happens when the components of a partially separable function have non-uniform contributions to the overall objective value. Contribution-Based Cooperative Co-evolution (CBCC) is a variant of CC that allocates the available computational resources to the individual components based on their contributions. CBCC variants (CBCC1 and CBCC2) have shown better performance than the standard CC in a variety of cases. In this paper, we show that over-exploration and over-exploitation are two major sources of performance loss in the existing CBCC variants. On that basis, we propose a new contribution-based algorithm that maintains a better balance between exploration and exploitation. The empirical results show that the new algorithm is superior to its predecessors as well as the standard CC

Decomposition for Large-scale Optimization Problems with Overlapping Components

In this paper we use a divide-and-conquer approach to tackle large-scale optimization problems with overlapping components. Decomposition for an overlapping problem is challenging as its components depend on one another. The existing decomposition methods typically assign all the linked decision variables into one group, thus cannot reduce the original problem size. To address this issue we modify the Recursive Differential Grouping (RDG) method to decompose overlapping problems, by breaking the linkage at variables shared by multiple components. To evaluate the efficacy of our method, we extend two existing overlapping benchmark problems considering various level of overlap. Experimental results show that our method can greatly improve the search ability of an optimization algorithm via divide-and-conquer, and outperforms RDG, random decomposition as well as other state-of-the-art methods. We further evaluate our method using the CEC'2013 benchmark problems and show that our method is very competitive when equipped with a component optimizer

Scaling Up Dynamic Optimization Problems: A Divide-and-Conquer Approach

Scalability is a crucial aspect of designing efficient algorithms. Despite their prevalence, large-scale dynamic optimization problems are not well-studied in the literature. This paper is concerned with designing benchmarks and frameworks for the study of large-scale dynamic optimization problems. We start by a formal analysis of the moving peaks benchmark and show its nonseparable nature irrespective of its number of peaks. We then propose a composite moving peaks benchmark suite with exploitable modularity covering a wide range of scalable partially separable functions suitable for the study of large-scale dynamic optimization problems. The benchmark exhibits modularity, heterogeneity, and imbalance features to resemble real-world problems. To deal with the intricacies of large-scale dynamic optimization problems, we propose a decomposition-based coevolutionary framework which breaks a large-scale dynamic optimization problem into a set of lower dimensional components. A novel aspect of the framework is its efficient bi-level resource allocation mechanism which controls the budget assignment to components and the populations responsible for tracking multiple moving optima. Based on a comprehensive empirical study on a wide range of large-scale dynamic optimization problems with up to 200 dimensions, we show the crucial role of problem decomposition and resource allocation in dealing with these problems. The experimental results clearly show the superiority of the proposed framework over three other approaches in solving large-scale dynamic optimization problems

Bandit-based cooperative coevolution for tackling contribution imbalance in large-scale optimization problems

This paper addresses the issue of computational resource allocation within the context of cooperative coevolution. Cooperative coevolution typically works by breaking a problem down into smaller subproblems (or components) and coevolving them in a round-robin fashion, resulting in a uniform resource allocation among its components. Despite its success on a wide range of problems, cooperative coevolution struggles to perform efficiently when its components do not contribute equally to the overall objective value. This is of crucial importance on large-scale optimization problems where such difference are further magnified. To resolve this imbalance problem, we extend the standard cooperative coevolution to a new generic framework capable of learning the contribution of each component using multi-armed bandit techniques. The new framework allocates the computational resources to each component proportional to their contributions towards improving the overall objective value. This approach results in a more economical use of the limited computational resources. We study different aspects of the proposed framework in the light of extensive experiments. Our empirical results confirm that even a simple bandit-based credit assignment scheme can significantly improve the performance of cooperative coevolution on large-scale continuous problems, leading to competitive performance as compared to the state-of-the-art algorithms

A review of population-based metaheuristics for large-scale black-box global optimization: Part A

Scalability of optimization algorithms is a major challenge in coping with the ever growing size of optimization problems in a wide range of application areas from high-dimensional machine learning to complex large-scale engineering problems. The field of large-scale global optimization is concerned with improving the scalability of global optimization algorithms, particularly population-based metaheuristics. Such metaheuristics have been successfully applied to continuous, discrete, or combinatorial problems ranging from several thousand dimensions to billions of decision variables. In this two-part survey, we review recent studies in the field of large-scale black-box global optimization to help researchers and practitioners gain a bird’s-eye view of the field, learn about its major trends, and the state-of-the-art algorithms. Part of the series covers two major algorithmic approaches to large-scale global optimization: problem decomposition and memetic algorithms. Part of the series covers a range of other algorithmic approaches to large-scale global optimization, describes a wide range of problem areas, and finally touches upon the pitfalls and challenges of current research and identifies several potential areas for future research

Solving Incremental Optimization Problems via Cooperative Coevolution

Engineering designs can involve multiple stages, where at each stage, the design models are incrementally modified and optimized. In contrast to traditional dynamic optimization problems where the changes are caused by some objective factors, the changes in such incremental optimization problems are usually caused by the modifications made by the decision makers during the design process. While existing work in the literature is mainly focused on traditional dynamic optimization, little research has been dedicated to solving such incremental optimization problems. In this work, we study how to adopt cooperative coevolution to efficiently solve a specific type of incremental optimization problems, namely, those with increasing decision variables. First, we present a benchmark function generator on the basis of some basic formulations of incremental optimization problems with increasing decision variables and exploitable modular structure. Then, we propose a contribution based cooperative coevolutionary framework coupled with an incremental grouping method for dealing with them. On one hand, the benchmark function generator is capable of generating various benchmark functions with various characteristics. On the other hand, the proposed framework is promising in solving such problems in terms of both optimization accuracy and computational efficiency. In addition, the proposed method is further assessed using a real-world application, i.e., the design optimization of a stepped cantilever beam

Benchmarking Continuous Dynamic Optimization: Survey and Generalized Test Suite

Dynamic changes are an important and inescapable aspect of many real-world optimization problems. Designing algorithms to find and track desirable solutions while facing challenges of dynamic optimization problems is an active research topic in the field of swarm and evolutionary computation. To evaluate and compare the performance of algorithms, it is imperative to use a suitable benchmark that generates problem instances with different controllable characteristics. In this paper, we give a comprehensive review of existing benchmarks and investigate their shortcomings in capturing different problem features. We then propose a highly configurable benchmark suite, the generalized moving peaks benchmark, capable of generating problem instances whose components have a variety of properties such as different levels of ill-conditioning, variable interactions, shape, and complexity. Moreover, components generated by the proposed benchmark can be highly dynamic with respect to the gradients, heights, optimum locations, condition numbers, shapes, complexities, and variable interactions. Finally, several well-known optimizers and dynamic optimization algorithms are chosen to solve generated problems by the proposed benchmark. The experimental results show the poor performance of the existing methods in facing new challenges posed by the addition of new properties

Efficient Resource Allocation in Cooperative Co-Evolution for Large-Scale Global Optimization

Cooperative co-evolution (CC) is an explicit means of problem decomposition in multipopulation evolutionary algorithms for solving large-scale optimization problems. For CC, subpopulations representing subcomponents of a large-scale optimization problem co-evolve, and are likely to have different contributions to the improvement of the best overall solution to the problem. Hence, it makes sense that more computational resources should be allocated to the subpopulations with greater contributions. In this paper, we study how to allocate computational resources in this context and subsequently propose a new CC framework named CCFR to efficiently allocate computational resources among the subpopulations according to their dynamic contributions to the improvement of the objective value of the best overall solution. Our experimental results suggest that CCFR can make efficient use of computational resources and is a highly competitive CCFR for solving large-scale optimization problems